
Calhoun: The NPS Institutional Archive 
DSpace Repository 


Theses and Dissertations 1. Thesis and Dissertation Collection, all items 


1976-06 

Analysis of program structure and error 
characteristics as applied to NTDS programs. 

Kirchgaessner, Michael 

Monterey, California. Naval Postgraduate School 


http://hdl.handle.net/10945/17667 


Copyright is reserved by the copyright owner 
Downloaded from NPS Archive: Calhoun 



DUDLEY 

KNOX 

LIBRARY 


Calhoun is the Naval Postgraduate Schools public access digital repository for 
research materials and institutional publications created by tire NPS community, 
Calhoun is named for Professor of Mathematics Guy K. Calhoun, NPS's first 
appointed — and published — scholar^ author. 

Dudley Knox Library / Naval Postgraduate School 
411 Dyer Road / 1 University Circle 
Monterey, California USA 93943 


http ://w w w. nps.edu/library 



ANALYSIS OF PROGRAM STRUCTURE AND 
ERROR CHARACTERISTICS AS APPLIED 
TO NTOS PROGRAMS 


Michael Kirchgaessner 



DUDLEY KNOX LIBRARY 

naval postgraduate schooi 

MONTEREY. CALIF. 93940 


NAVAL POSTGRADUATE SCHOOL 

Monterey, California 



THESIS 

ANALYSIS OF PROGRAM STRUCTURE AND 
ERROR CHARACTERISTICS AS APPLIED 
TO NTDS PROGRAMS 

by 

Michael Kirchgaessner 

June 1976 

Thesis Advisor: N. F. Schneidewind 


Approved for public release; distribution unlimited. 


117497 1 








SECURITY CLASSIFICATION of this PACE (Whmn Dmtm Sntmtmd) 


REPORT DOCUMENTATION PAGE 

READ INSTRUCTIONS 

BEFORE COMPLETING FORM 

1. REPORT NUMBER 

2. GOVT ACCESSION NO. 

3. RECIPIENT’S CATALOG NUMBER 

4. TITLE ftnd Submit) 

Analysis of Program Structure and Error 
Characteristics as Applied to NTDS 
Programs 

5. TYPE OF REPORT A PERIOD COVERED 

Master's Thesis; 

June 1976 

A. PERFORMING ORG. REPORT NUMBER 

7. AUTHOR/*; 

Michael Kirchgaessner 

A. CONTRACT OR GRANT NUMBER/*; 

9. PERFORMING ORGANIZATION NAME AND AOORESS 

Naval Postgraduate School 

Monterey, California 93940 

10. PROGRAM ELEMENT, PROJ ECT, TASK 

AREA A WORK UNIT NUMBERS 

II. CONTROLLING OFFICE NAME ANO AOORESS 

Naval Postgraduate School 

Monterey, California 93940 

12. REPORT DATE 

June 1976 

13. NUMBER OF PAGES 

105 

U. MONITORING AGENCY name a AOORESS/// dittmrmnt /mm Controlling Ottlcm) 

Naval Postgraduate School 

Monterey, California 93940 

IS. SECURITY CLASS, (at thlm rdport) 

Unclassified 

IS*. DECLASSIFI CATION/DOWNGRADING 

SCHEDULE 

IS. DISTRIBUTION STATEMENT (at Ihit Report) 

Approved for public release; distribution unlimited. 

17. DISTRIBUTION STATEMENT (of thm mbmtrmct mntmrmd In Block 20, It dlttmtmnt from Rmport) 

18. supplementary NOTES * 

19. KEY WORDS (Continum on tmrmrmm ml do It nmcmmmmry mnd Identity by block numbmt) 

Program structure 

Program complexity 

NTDS programs 

Program testing 

20. ABSTRACT (Contlnum on rmrmrmm mi dm It nmcmmmmry mnd Idmntlty by block membmr) 

A simulation model for the evaluation of program structure 
and error detection has been applied to the analysis of 
selected parts of NTDS programs. The simulation results were 
used to establish the relationship between program structure 
and measures of program complexity. This information would 
be used for the design and testing of software. 


DD | jAN *73 H73 EDITION OF I NOV 51 IS OBSOLETE 

(Page 1) S/N 0102-014* 6601 I 


SECURITY CLASSIFICATION OF THIS PAGE (Whmn Dmtm Intmrmd) 


1 




























faCUtflTV CLASSIFICATION of This PtGEl'W^.n r>... Enltr.d 


DD Form 1473 

1 Jan 73 __ 

—/ N 0102-014-6601 security classification of this pagei'**** a«<« 


2 







ANALYSIS Cl PRCGRAM STRUCTURE AND ERROR CHARACTERISTICS AS 

APPLIED TC NTDS PROGRAMS 


by 


Michael Kirchgaessner 
Lieutenaht-Ccmmander 
Federal German Navy 


Submitted in partial 
requirements for 


fulfillment of the 
the degree cf 


MASTER CF SCIENCE IN COMPUTER SCIENCE 


from the 

NAVAL POSTGRADUATE SCHOCI 
June 1976 





WY KNOX LIBRARY 

postgraduate SCHUOt 

MONTEREY. CALIF 93940 


ABSTRACT 


A simulation model for the ev 
structure and error detection has b 
analysis cf selected parts of 
simulation results were used t 
relationship between program struct 
program complexity. This informati 
for the design and testing cf softw 


aluaticn of prog 
een applied tc 
NTDS programs, 
c establish 
ure and measures 
on would be u 
are. 


ram 

the 

The 

the 

cf 

sed 


4 



TABLE CE CONTENTS 


I. INTBCDUCTIO N. 7 

II. DEFINITIONS. 10 

III. MODERN PROGRAMMING TECHNIQUES. 12 

A. MODULAR PROGRAMMING. 12 

E. STRUCTURED PBOGBAHMING. 14 

IV. TEE PECELEM CB PBOGBAM COMPLEXITY. 16 

V. ESECE DETECTION SIMULATION MODEL. 18 

A. GENERAL. 18 

B. PECGEAM REPRESENTATION. 18 

C. CUERENT STATUS OF THE SIMULATION PBOGBAM. 19 

1. Input Variables.... 19 

2. Input Formats. 20 

3. Limitations. 21 

4. Program Listing. 21 

VI. ANALYSIS OF NTDS PSOGEAMS. 22 

A. GENEEAL. 22 

I 

B. DESIGN CF NTDS PROGRAMS. 22 

1. Modular Design. 22 

2. CS-1 Language. 23 

C. DIRECTED GRAPH CONSTRUCTION. 24 

D. EEECB DETECTION SIMULATION. 27 

E. RESULTS OF THE ANALYSIS. 28 

1 . Module One. 29 

2. Module Two. 31 

VII. USE CE TEE RESULTS..... 34 

A. AIDS FOE SOFTWARE DEVELOPMENT. 34 

E. FUTURE WORK. 34 

VIII. SUMMARY AND CONCLUSIONS. 36 

IX. ACKNOWLEDGEMENTS. 37 

Appendix A: ERROR DETECTION SIMULATION PROGRAM. 38 


5 


















































Appendix E: LIST CF EVALUATED PROGRAM STRUCTURES. 47 

Appendix C: DIRECTED GRAPHS... 53 

LIST OF 5EFEBENCES. 102 

INITIAL DIS1EIEUTICN LIST. 104 


6 

































































I. 


INTRODUCTION 


When is a program considered to be trivial? One answer 
to this guesticn heard very often is '’When it contains no 
bugs". although this statement might be Questionable, the 
converse is true, as there are few nontrivial programs that 
do net certain bugs. As the author of a critical and 
fundamental study of program design states: "...These tugs 
can never he completely exorcised in any program over some 
critical decree cf complexity. Six months or even seven 
years after 'final debugging' errors crop up inevitably in 
the best cf programs."[4 ]. This is a fact one has tc live 
with, and there are .only two things one can do about it: 
First tc reduce the possibilities for bugs by careful design 
and use cf modern programming techniques, second tc devise 
careful testing techniques to detect and locate the bugs 
still remaining in tie program. 

Fig. 1 shews the relationship between hardware and 
software cost in the U.S. during the period from 1955 to 
19£5. Cue to the fact that the software cost continues to 
rise and that about 50X of this cost is for testing and 
integration cf a system [7], it is important to obtain a 
realistic assessment of how much effort has tc be spent to 
test the newly designed program based on its size, structure 
and characteristics. If one is able to determine ir the 
design stage the best possible structure with respect to the 
error detection capabilities, then bugs can be avoided and 
testing will be reduced. Also early in the development cf a 
project a realistic allocation of coding and testing 
resources could be made. 
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Figure 1 - SOFTWARE COST TREND IN THE D.S. 

[ Datamation, Sept. 1974, pg. 75] 

In order to address these problems, a Software Error 
Detection Simulation Model has been developed [7,10]. This 
model was was used to identify program complexity measures 
which were correlated with error detection. Naval Tactical 
Data System programs were used for this purpose. 

The structures of these NTDS-programs have been analyzed 
(see Chapter 71) and put into the form of directed graphs. 

The data gained from the directed graph representation 
were used as inputs for the Error Detection Simulation 
Model. The results gained and the conclusions and 

recommendations drawn from these results are shown in 
Chapter 711. For reasons of security, the programs or the 
parts of them are not identified by names. Instead, a 
sequential number scheme for identifying the programs has 
been employed. 
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This work is part of a research effort sponsored ty tne 
N AC C to get software evaluation aids which provide an 
economical assessment of the design and testing effort 
needed for the development of avionics and ether complex 
software prefects. 


Eecause it is 
debugging can be 
techniques in the 
chapter shows the 
to the problem of 


felt that efforts in testing and in 
mere successful if one employs modern 
production of programs, an introductory 
relevance of modern programming techniques 
program testing and maintenance. 
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II• DEFINITIONS 


There was originally a lack of commonly used definitions 
for program testing. Only recently has a "definitional 
framework" emerged and very good program testing definitions 
are found in Ref. 8, pg. 7 - 14. In order to be consistent 
and to specify the meaning of keywords within this thesis, 
the fcllcwinc definitions have been adopted: 


1 • Etesian; St ruc tur e 


The structure of a program is a description of the 
underlying logic and data flow as represented in the 
form cf a directed graph with its set of nedes and 
edges (arcs) . 


2• Reachability Index 


Reachability index is 
possibilities to get to a 
ever all nodes of the 
computed with the formula 


a measurement 
specified node, 
directed graph. 


o i the 
computed 
It is 


R 



to node 


(i) • 


Debugging 


Debugging is the action one takes to locate and 
correct a known or detected error in a program. 
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4 . 


Testing 

Testing is the action to check whether a program 
meets its specifications and to establish the 
presence of errors in it. 

5. life c ycle of a prog ram 

Ihe life cycle of a program consists of the 
fcllcwing phases: 

- design 

- Coding 

- Eebucging 

- Testing 

- Production and maintenance. 




















































III. MODERN PROGRAMMING TECHNIQUES 


Two recent developments in the theory and practice of 
software development are addressed here as important because 
they are relevant not only for the actual writing of the 
code of the program, but also to debugging, testing, and 
integrating software systems as well, namely the advent of 
modular and structured programming. The advantages of these 
technigues are obvious for the programmer when he develops 
his program. Programs written using these technigues are 
easier to read and to understand as far as the flew of the 
logic is concerned. Also, the tester can better understand 
the logic of a program when these technigues are employed. 
Furthermore, it has been proposed for structured programs to 
eliminate flowcharts as media of communication [13], sc it 
is necessary tc understand how much testing, integration and 
maintenance of software are influenced by this development. 


A. MCDUIiE EECGEA MMING 


Modular programming is a system to develop programs as a 
set of interrelated individual units (called modules) which 
later can be linked together tc form a complete program [9]. 
Thus modular programming is not simply splitting up a 
program into several parts (subroutines), but rather 
dividing the software according to the functions tc be 
performed. The designer faces the one crucial problem which 
will determine success or failure, namely to specify 
completely and carefully the interface between the 
individual modules. 
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Modules as individual program units should have the 
following properties: 

(1) Cre nodule should perforce only one basic function 

(2) The size of a module should be such that it is 
easily understood and contains crly a moderate 
amount of code 

(3) fl module should be designed in such a way that it 
has only a few control or data paths 

(4) Cne nodule should process only a small amount of 
data. 


The design of programs in this way leads not only to 
cleaner and more productive coding but also to easier and 
more flexible testing. The advantages with respect to 
debugging and testing show up in several ways. Single 
modules can he debugged and tested independently from the 
ether modules or the main (driver) program. Furthermore, if 
the modules are snail enough, extensive testing generally 
assumed as impossible with the exception of very trivial 
programs, can become manageable. This in turn leads tc more 
reliable programs. If all modules of a software project can 
be tested extensively, a highly reliable program can be 
produced. Iven if one falls short of this goal - and this 
happens ir most cases due to the very large number of 
possible inputs and program paths - the final program will 
be mere reliable and more thoroughly tested than a 
ncn-mcdular program. The possibility of testing modules 
individually provides for better (mere economical) 
allocation of testing resources, because cne does net have 
tc wait until the whole program has been completed. However, 
to test individual modules, special test-routines are needed 
as drivers and if ether modules must interact, dummy modules 
must be created if the real modules are not yet available or 
not yet tested. 


13 



















One final point in favour of modular programming has to 
be made: Normally, no production program is completed until 
the day when it is no longer used, i.e. every running 
production program has to be maintained and adapted tc new 
considerations and situations. Because of the simplicity of 
the overall crgani2ation of modular programs this software 
maintenance is alleviated since interactions between modules 
are more easily understood; hence, the effect of program 
changes is easier to identify. Also only the modules 
affected by the change have to be tested (together with the 
main program and interacting modules). 


B. SIROCIOEIE PRCGEAKMING 


Having coded a program in the above described 
modularized fashion; there is still room for improvement. 
Since Eijkstra's famous letter to the editor of the 
Communications of the ACM in which he proposed to eliminate 
GC-TO statements [5]* the concept of Structured Programming 
has evolved and led tc further simplification of the coding 
process. 

Simplification means here not that the actual code is 
easier tc write - although this might be the case tcc for a 
programmer who is familiar with the concept and can think in 
these terms - but the code produced and the control secuence 
of the finished program is simpler than in a non structured 
pregram. ihis simplification has been theoretically 
demonstrated by Boehm and Jaccpini as early as 1966 [3]. 
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Although there are as many interpretations of what 
Structured Programming is as there are authors on this 
topic, the following features are essential and commcn to 
this concept: 

(1) ICP-ICWN Desggn, i.e. the design starts at a very 
general level and proceeds stepwise to the specific 
and detailed tasks 

(2) Modular Design 

(3) limited possibilities to control the logic flew of 
the program, namely cnly 

* sequential 

* conditional: IF - THEN - ELSE 

* iterative: DO - WHILE 

statements are allowed. 

Whereas the so called block-structured languages like 
ALGOL cr El/I lend themselves to this form cf coding 
(although GC-IO statements are provided 1 by the language), 
even in ECETEAN the implementation of seme of the basic 
principles cf Structured Programming is possible if the 
programmer concerned with a structural flew cf his program 
cheeses the branching caused by unavoidable GC-TO 
statements carefully. 

Eaker [1] shows that the application of Structured 
Programming combined with the "Chief Programmer Team Method" 
of organising a software project [2] can bring measurable 
improvements in software development, in the coding as well 
as in the debugging and in the testing stage. Due to the 
fact.that Structured Programming implies Modular Programming 
the same advantages hold here too, i.e. the software is 
easier tc test and tp maintain after release. 
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IV. 


THE PROBLEM CP P ROGR AM COMPLEXITY 


The impact of the programming techniques described above 
cn the economic development of reliable and maintainable 
software is directly related to the problem of program 
complexity. There is so far no generally adopted definition 
of what program complexity really means. The definition is 
dependent or the context in which one wants to examine 
program complexity. Here complexity is defined as structural 
properties of a program that affect the ability to detect 
errors. 

Under the condition that the structure of a program is 
described by a directed graph, the following criteria can be 
used to measure its complexity: 

1. Number of nodes 

2. Number of arcs 

3. Number of possible paths through the program 

4. Number of source statements 

5. Averacs path length (source statements per path, arcs 
per paths) 

6. Reachability index 

7. Fullness index (ratio of actual to maximum number of 
arcs) . 

Although Mills in his contribution to Ref. 8 generates the 
idea of equating program complexity with the difficulty of 
understandinc a prpgram and justifies this approach with 
"-‘..the frustration of concocting and demolishing more 
simple minded direct ideas, such as counts of branches, data 
references, etc.' 1 , his approach does not help to get a real 
measurement of complexity such that one is able to make a 
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quantitative stateliest how complex a program is. It seems 
that the important point is tc relate program complexity to 
the problem area one pursues. The analysis cf NTDS-Pcgrams 
has given insight in methods to measure complexity with 
respect tc problems of program design and testing. 
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V. 


ERROR DETECTION SIMULATION MODEL 


A. GENEEAL 


A Software Error Detection Simulation Model was 
originally developed by T.F. Green in his M.S. Thesis [7] 
and subsequently mpdified by professor G.T. Howard cf the 
Naval Postgraduate School. Written in FORTRAN it was 
designed to run on the IBM 360/67 computer of the Naval 
Postgraduate School. Originally it had been tested acainst 
hypothetical and actual programs. It was shewn that 
simulation cf error detection was feasible and that 
information could be obtained on the relationship between 
error detection and program complexity. However, it was 
necessary to perform additional model feasibility tests by 
using the model on a large number of actual programs. In 
the process cf testing some cf the original features had to 
be removed arc provisions had to be made for cases of 
pregram behaviour which were unexpected at the time cf the 
simulation pregram design. A detailed description of the 
model with its specific assumptions and capabilities is 
found in Eef. 10, pg.. IV-5 - IV-39. 

E. PRCGEAM EEERESENTATION 


The prerequisite for the use of the simulation model is 
to get the structure of a program that has to be tested in 
the form cf a directed graph. A directed graph is a 
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convenient means tc show the structure of programs. It is 
suitable for showing the control flow in a program, measures 
of complexity can be derived from this kind of 
representation. In addition, the "control flow graph" as 
this composition of structures is sometimes called, is also 
very useful for determining the execution time of a 
structure on a machine. This representation of program 
structures also simplifies the representation of large and 
complex programs because these programs can be broken up in 
logical segments (modules, procedures, subroutines etc.), 
and the segments can be tested separately from the ether 
parts of the program. 


C. CUBE Z NT STATUS 0<F THE SIMULATION PEOGHAM 


1 . 

Input 

Variables 





Ihe 

following input 

variables have to 

be used for 

the si 

mu laticn: 




a . 

MINE C T 

designates the 

number of inputs 

within each 



replication. 




b. 

NUMOUT 

is the number 

of 

replications 

(number of 



pathsl within every 

repetition. 


c . 

NEEPE1 

is the number 

cf 

reseedings 

with errors 



(repetitions) . 




d. 

MEANLN 

designates the 

mean arc length if the arc 



lengths are selected 

at random by 

the program 



and are not read 

in. 



. e. 

MEANEE 

designates the mean 

number cf 

instructions 



between errors. 




f . 

N 

is the number of 

nodes within the 

structure. 

g- 

Input 

for the Adjacency 

Matrix is dene in 

a shorthand 
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notation: 

For every node with the exception cf the 
there is one data card which contains i 
about this node in the following 
Identification of the node, number cf arcs 
from this node, identification numbers of 
tc which the arcs go. 

h. Input for the matrix of arc lengths 
similar to that for the adjacency matrix: 
only as the identifiers for receiving node 
(identifier, number of statements on this a 
be provided. 

i. Input tc plant errors in arcs instead cf le 
program seed them at random: Input as for 
arc length, but the number cf errors on thi 
tc be specified instead of the number cf st 

j. MCGT specifies the desired output: 

0 = Summary output 

1 = Extensive output (NUMOUT * NBEPE1 < 25) 


2. Input Eormats 


Ihe input formats are as follows: 

First data card: (615) 21INPUT, NUMOUT, NREPE 

MEANEB, ii. 

Second and following cards: adjacency matrix 
followed by delimiter-card: 59 in columns 4 and 5. 

Input cards for matrix of arc length (cptio 
7(I5,F5.C); followed by delimiter-card: 99 in col 
5.. .. 

Input to seed errors manually (optional) : 1615; 

delimiter-card: 99 in columns 4 and 5. 


last nodes 
nfor mation 
sequence: 
emanating 
the nodes 

(optional) 
Instead, 
s the pair 
rc) has to 

ttinc the 
matrix of 
s arc has 
atements. 


T, KEABIM, 

, (1615); 

nal): 215, 
umns 4 and 
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Last data caid (output specification): 15. 

Note that all delimiter cards are not optional. 

3. limita tion s 

This simulation program is currently restricted to 
accomodate a maximum number of 30 nodes. The execution time 
for simpler structures (about 10 - 15 nodes) is within a 
five minute time limit. Larger and more complex structures 
with more redes and possible paths through the structure 
reguire a 30 minute time frame for the execution of one 
simulated input in 100 replications and 100 repetitions. 

An extension of the limits of the program to 
accomodate larger structures seems to be impractical because 
of the fast rise of memory space and execution time 
reguired. 

4. Frog ram Listing 

A listing of the current error detection simulation 
program as it was used for the analysis of the NTDS-prcgrams 
is fcuDd in Appendix A. 
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VI. AN ALY SIS OF NTDS PHCGFAMS 


A. GENEFAL 


In order tc demonstrate the practicality of program 
analysis using the Error Detection Simulation Model, Naval 
Tactical Data Systems Programs have been analyzed by 

1. describing the structure by converting the programs 
into the form of directed graphs 

2. running these structures on the error detection 

simulation model and 

3. evaluating the simulation results with respect to 
measures of program complexity. 


B. DESIGN CE NTDS F-RCGEAMS 


1. Scdular Design 

Ihe design of NTDS programs is characterized by 
Modular Ire cramming, both in general and in detail, and the 
modular design is a characteristic of the hardware as well. 
Alsc the actual implementation of every NTDS installation 
consists of hardware and software building blocks that are 
composed tc fit exactly the need of each installation. 

Although NTDS programs are really programmed in a 
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modular fashion, the term "module" does not have the same 
meaning as usual. Mpdule usually refers to basic building 
blocks that are farts of the program, whereas NTDS programs 
are composed of subsystems. The NTDS-"Modules" in turr are 
divided up iD parts which correspond to the 
"mcdule"-definiticn of Modular Programming. In NTDS 
terminology these parts are called procedures. NTDS modules 
perform complex tasks such as tracking, display etc. They 
certain a medium to large number of dependent procedures. 
These procedures perform the basic functions intended in 
Modular Programming such as checking track properties. 
Throughout this discussion, "module" is used as in the NTDS 
system, namely as a complete subsystem for performing 
complex tasks. 

The modular approach is imbedded in a stringent 
hierarchical system which is controlled by the priorities of 
the tasks to be performed. The levels of hierarchy are 
applied to the modules in such a way that only major 
subprograms which are designed to execute distinctive tasks 
can communicate with each ether, whereas the procedures 
within the modules can only communicate according to the 
level of hierarchy they belong to, with the exception of 
calls to certain system routines. 

2. CS-J langu ag e 

The NTDS programs are written using the CS-1 high 
level language compiler [6]. This language has the advantage 
that it is well suited to the application area, Eamely 
tactical pregrams which run under severe constraints 
regarding time and memory space availability. Tables are 
searched in a very effective way, and another interesting 
feature is that assembly code can be interspersed within the 
high level code of the program. This fact gives the 
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programmer a powerful means for controlling the hardware 
which in turn facilitates the production of effective cede. 


C. EIREC3EE GEAEH CONSTRUCTION 


In order to ottain the desired statistics and to analyze 
the data- anc control flew of a single NTES-program , the 
following method has been developed and used: 

1. One complete module from an existing and currently 
operating NIBS program has been put into the form of 
a directed graph. The module has teen decomposed 
into the procedures it contains, and every procedure 
is treated separately. Due to the modular design 
theugheut the program, no logical difficulties arise 
here, because every procedure has only one entrance 
and one exit point, i.e. the interface for 
interacting procedures within the module is uniguely 
defined. For each procedure the directed graph and 
the adjacency matrix have been constructed. As 
quantitative measurements the number of nodes, arcs, 
paths, loops, source statements, machine 
instructions, source statements per arc, and machine 
instructions per arc have been compiled. 

2. lhe same work was done for randomly selected 
procedures from one other important module of the 
same program in order to obtain comparative results 
and to relate the reported number of errors to the 
different modules. 
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. Fcr tfc€ construction of the directed graphs anc the 
gathering of the several statistics the following 
assumptions have been made: 

a. Nodes are associated with 

(.1) Procedure entrance and exits 

(2) IF-statements (decision points) 

(3) Points where paths merge 

(4) Procedure calls within the module 

(5) Beginning and ending cf loops 

b. All nodes within the module are distinct. 
They have individually assigned numbers (seme 
nodes are indicated as "dummy" nodes), and 
they are counted only once, namely in the 
procedure they belong to. 

c. Entrance and exit nodes of a called procedure 
are regarded as "transient" nedes within the 
calling procedure, and one "transient arc" 
connects both transient nodes. This 
transient arc represents all the arcs inside 
the called procedure. The transient arcs are 
indicated in the drawings by a dashed line. 
Transient nodes have either the number cf the 
entrance node of the called procedure or they 
are denoted by letters tc distinguish them 
from the original nodes of the corresponding 
procedure. 

d. The Length of every arc is indicated as the 
number of source statements cr the number of 
machine instructions respectively. Ie the 
analysis the number cf source statements has 
been used because programs are normally 
written in a high level language and this is 
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the point where errors are introduced into 
the program. 

4. Normally, the numbers of both source statements and 
machine instructions have been counted in the arc 
where the statements appear. However, because 

Il-statements and procedure calls result in 

branching, they have been counted in the arc leading 
tc the corresponding node. Whereas for the counting 
cf machine instructions, it would be possible in the 
case cf an IF-statement to split the instruction 
sequence according tc the arcs emanating from the 
decision point, this is not feasible for the source 
statement which contains the elements of both arcs 
emanating from it; it cannot be split. 

The structures obtained from both modules anc the 
compiled statistics are found in Appendix E. The following 
figure shews hew tc read the structure diagrams: 



Figure 2 - EXAMPLE OF PROGRAM SIEUCTOR E 
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c. 


EBROB DETECTION SIMULATION ON THESE STRUCTURES 


The structures which were converted into directed graphs 
for Module Ce 6 were screened to determine their suitatility 
for errcr detection simulation. It would have teen 
desirable to select a random sample of the structures. 
However, it was necessary to choose structures which wculd 
net reguire excessive amounts of memory space and CPU time 
during the simulation. In addition, the structures were to 
have at least two or more paths. In the case of Module Two 
it was leasable tp use a random sample because a high 
percentage of the structures fell within the memory space 
and the CEU time limitations of the model. 

The input cata for the simulation were taken from the 
actual programs, including the number of source statements 
for every arc. The recorded number of errors per module was 
used to calculate the mean number of instructions between 
errers, which is used for seeding errors in the simulation 
model. Seeding the errors was done randomly by the 
simulation program. However, it was provided that no errors 
were seeded at arcs containing zero instructions (control 
arcs) . 

The simulation was run with one input, 100 replications 
and 100 repetitions (reseedings), and the average number of 
errors feund by o.ne input was obtained. although sene of 
the structures were small, and a higher number of 
repetitions and replications could have been run, the same 
simulation parameters were used for each structure in order 
to obtain comparable results. 
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E. 


RESULTS Cl THE ANALYSIS 


From the average of errors found by one input in each 
procedure the average percentage of errors found against the 
errors expected within the procedure was obtained. These 
results were plotted against various complexity measures, 
e.g. the cumber of paths. Although the results varied 
somewhat between tie modules, it was possible to establish 
relationships between structural properties and error 
detection capabilities. 

The differences in results between modules can be traced 
to several factors: 

1. Different sample sizes: 

From Module One 32 procedures were used, 16 
piecedures were randomly selected frem Module Two. 

2. The different size of the modules: 

Module One had 97, and Module Two had 155 procedures. 

3. Differences in program design and programming style: 
Module Two was modularized to a much larger extent 
than Module One. It was hard to find a sufficient 
number cf paths within randomly selected procedures 
of Module Two. 

4. Different number of reported errors: 

Although Mcdule Two was 1.6 times larger than Module 
One, it had only about two-thirds the number of 
errors. 

The following diagrams show the percentage of average 
errors feund against the expected number cf errors fer the 
structures cf both modules. 
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Figure 6 - PERCENTAGE ERRORS FCUND VS. SOURCE STATEMENTS 
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I 


The 

curves shewn represent 

ex 

pen ential 

approximations 

to the datapeints according 

to 

the 

formula 

y=a*e**(-t*x) 

which was found to represent 

the 

rel 

atienship 

best. 3 least 

Sguare fit was used. 





311 diagrams show seme relationship retween error 
detection and complexity. Module One with its larger sample 
size shews this relationship more than Module Two fer the 
number of paths. This seems logical because a large number 
cf paths reduces the ability to detect errors in a program. 
It appears that the number of paths could be used as a 
measure cf program complexity for design and testing 
purposes. 

In order to rank the approximations, a squared error 
factor has teen computed for every complexity measure as 
fellows: 



Error 

Factor 


Mod. 1 

Mod. 2 

Nodes 

7337 

4430 

Arcs 

684 1 

3933 

Paths 

4995 

4666 

£.stmts. 

6575 

1808 

This computation 

sho-ws that fc 

r Modul 


paths is the best approximated complexity 
method used. 3nother interesting aspect found 
approximated relationship between percentage 
and the numter cf source statements in module 


the n 
measu 
was 

cf err 
Two. 


umber 
re ty 
the 
ers f 


of 

the 

well 

cund 
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VII. USE Of THE RESULTS 


A. AILS ECR SCETWAHE DEVELOPMENT 

This method cf program analysis provides the so 
manager with information for selecting structures easi 
the design process. He can choose the least c 
structure which Mill satisfy project require 
Furthermore, after a project has been coded and is d 
testing, he can make realistic " assessments ccncernin 
effort which will be needed for program testi 
considering factors such as 

1. expected complexity of the project 

2. chcice cf the programming techniques used 

3. organization and experience of the programming 

4. available manpower and computer time for t 
pur poses. 

B. FUTURE &CEK 

The analysis dene on the NTDS programs and the r 
obtained fer the measurement of program comp 

represents a modest contribution to the field of so 
engineering. But being far from complete or exhausti 
following steps should be taken in order to 

additional validation of the analysis process. 
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1 • Jurthej: evaluation of NTDS-Modules 

Additional NIDS modules should be evaluated in 
to obtain larger sample sizes. It is realized tha 
evaluation process for the important modules is ver 
consuming. However, the more important modules are 
mere freguently and will, in most cases, have a longer 
history, which will provide valuable data for comp 
with simulation results. 


2. Jyaluati.cn of stru ctured pr o gr ams 


It would 
the evaluation of 
perform the same 
into a structured 
structured progra 
error detection. 


be of interest in 
the NTDS-prccedures 
functions but are 
programmed form. It 
ms would perform 


this respect to c 
with procedures 
rewritten and con 
is expected tha 
better with resp 


order 
t the 
3 time 
used 
error 
arison 


empare 
that 
verted 
t the 
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VIII. S UMMA RY AND CONCLUSIONS 


A method to define and analyze program structures has 
teen presented. All measurements obtained were based on the 
description of the program structure in the form of a 
directed graph ana the use of the error detection simulation 
model. Ihis method has been used to analyze the procedures 
from two N1ES modules. It was beyond the scope of this 
effort to obtain comparative results between this experiment 
and the actual error history of the programs. However, it 
was possible to obtain an initial quantitative assessment of 
measures of complexity. 

By using this method to check program structures in the 
design phases it should be possible to produce programs with 
structures that are less complex and therefore easier and 
more economical to test and maintain. Also the method could 
be used during the test phase as a means of assigning test 
resources. 
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APPENDIX A 


ERROH DETECTION SIMULATION PROGRAM 
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Software 
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May 1976 
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rogram listed on the following pages s 
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APPENDIX 3 


IIST CF EVALUATED PROGRAM STRUCTURES 


This list gives all the statistical data gathered from 
the conversion of the procedures of the NILS modules into 
the form of directed graphs. The abbreviations read as 
fellows: 


PNR 

N 

A 

P 

L 

Ss 

Mi 


Procedure 
Number of 
Number of 
Nunber of 
Number cf 
Number of 
Number of 


number within the module 

nodes (including transient nodes) 

arcs (including transient arcs) 

paths 

loops 

source statements 
machine instructions 


SA Source stmts./arc 

MA Machine instr./arc 
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APPENDIX C 


DIRECTED GRAPHS 


Cn the following pages the structures of all the 
procedures are listed that were used as input data for the 
Error Detection Simulation Model. In addition tc the 
complexity measures used also listed are the results 
ottained from the simulation, the average number of errors 
found with 1 input, ICO replications and 100 repetitions, 
and the percentage of expected errors detected. 

Differently tc the sample structure shown in Fig. 
2, the number cf statements is indicated in the following 
graphs ODly fcr arcs with ncnzerc instructicns. 


The count for the number of nodes and the number of 
arcs includes the transient nodes (designated by letters) 
anc the transient arcs (dashed lines) because they must be 
included into the inputs for the Error Detection Simulation 
Program. 
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Module: 1 


Procedure No.: 2 



Nunter of nodes: 

14 

tiumfcer of arcs: 

23 

Nunter of paths: 

26 

Nuich’er of source stmts.: 

37 

Average error found: 

0.3144 

Percentage errors found: 

17.84 
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Module: 1 


Procedure No.: 8 



Number of nodes: 

13 

Number of arcs: 

1 4 

Number of paths: 

3 

Number of source stmts.: 

10 

Average error found: 

0.2523 

Percentage errors found: 

52.S8 
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Module 


1 


Procedure No.: 11 



Nunfcer of nodes: 6 

Nuater cf arcs: 8 

Nunfcer of paths: 5 

Nuiiier cf source stmts.: 8 

Average error fcund: 0.1974 

Percentage errors fcund: 51.82 
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Module 


1 


Procedure No.; 14 



Hunter of nodes: 

Number of arcs: 

Number of paths: 

Number of source stmts.: 
Average error found: 
Percentage errors found: 


6 

7 

4 

9 

0.2586 

60.34 
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Module: 1 


Procedure No.: 19 



Number of nodes: 

19 

Number of arcs: 

26 

Number of paths: 

7 

Number of source stmts.: 

45 

Average error found: 

0.2885 

Percentage errors found: 

27.54 
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Module: 1 


Procedure No.: 


2 2 



Nuiifcer of codes: 

25 

Nuiifcer of arcs: 

30 

Nuiifcer of paths: 

11 

Nuiifcer of sccrce stmts.: 

30 

Average error found: 

0.4105 

Percentage errors found: 

28.74 
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Module: 1 


Procedure No. 



Number of nodes: 12 

Number cf arcs: 12 

Number of paths: 2 

Number cf source stmts.: 8 

Average error found: 0.2324 

Percentage errcrs found: 61.01 












Module: 1 Procedure No.: 


4/16 


Number of nodes: 

17 

Number of arcs: 

19 

Number of paths: 

4 

Number of sccrce stmts.: 

32 

Average error found: 

0.6400 

Percentage errors found: 

42.00 



28 
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Module; 



Hunter of nodes: 

28 

Hunter of arcs: 

32 

Nuiter of paths: 

9 

Nunter cf source stats.: 

47 

Average error found: 

1.3946 

Eercentace errors found: 

62.31 


29 
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Hcdule 


1 


Procedure No.: 30 



Number of nodes: 

7 

Number of arcs: 

10 

Number of paths: 

5 

Number cf source stmts.: 

10 

Average errcr found: 

0. 1649 

Percentage errors found: 

34.63 
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Module 


1 


Procedure No. : 34 



Nuater cf nodes: 16 

Hunter cf arcs: 17 

Nuater of paths: 3 

Nucter cf scurce stmts.: 5 

Average errcr found: 0.5465 

Percentage errors found: 76.51 
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Module 


1 


Procedure No. : 


35 



Hunter of ncaes: 

14 

Number cf arcs: 

17 

Number of paths; 

3 

Number of sccrce stmts.: 

14 

Average error found: 

0.3576 

Percentage ericrs found; 

5 3.64 


65 










flcdule: 1 


Procedure No.: 36 



Nuiiter of ncces: 21 

Number cf arcs: 26 

Number of paths: 3 

Nuafcer cf scurce stmts.: 31 

Average errcr found: 0.5203 

Percentage errors found: 35.25 
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Module 


1 


Procedure No 


39 



1/1 


Nunter of 

ncdes: 

17 

Number of 

arcs: 

25 

Nunber of 

paths: 

10 

Number cf 

sccrce stmts.: 

17 

Average error found: 

0-2637 

Percentage 

errors found: 

32.57 
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Module 


1 


Procedure No.: 


44 



Number of nodes: 27 

Number of arcs: 30 

Number of paths: 7 

Number of source stmts.: 21 

Average error found: 0.3554 

Percentage errors found: 35.54 
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Module: 1 


Procedure No 


47 



Number of nodes: 19 

Number cf arcs: 20 

Number cf paths: 4 

Number cf source stmts.: 12 

Average error found: 0.4231 

Percentage errors found: 74.04 
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Module: 1 


Procedure No. : 48 



Nuirter of nodes: 23 

Number cf arcs: 26 

Nuater of patbs: 7 

Number cf source stmts.: 13 

Average error found: 0.3287 

Eercentace errors found: 53.1C 
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Module: 1 


Procedure Ho.: 49 



Number of nodes; 

15 

Number of arcs: 

18 

Number of paths: 

7 

Number of source stmts.; 

19 

Average error found: 

0.2217 

Percentage errors found; 

24.50 
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Module 


1 


Procedure No.: 


53 



Nunter of nodes: 

11 

Nunter of arcs: 

18 

Nunter of paths: 

9 

Nunter of source stmts.: 

11 

Average error found: 

0.1876 

Percentage errors found: 

35.81 
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Ccdule: 1 


Procedure No.: 57 



Number of cedes: 30 

Number of arcs: 36 

Number of paths: 14 

Number cf setree stmts.: 26 

Average error found: 0.2910 

Percentage errors found: 23.50 
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Module: 


1 


Procedure No.: 60 



Number of 

oc ces: 

28 

Number cf 

arcs: 

37 

Number of 

paths: 

18 

Number cf 

sctrce stmts.: 

24 

Average errcr fcund: 

0.3336 

Percentage 

eircrs found: 

29.19 
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Procedure No.: 75 


Number cf arcs: 28 

Number of paths: 8 

Number cf source stmts.: 20 

Average errcr found: 0.5433 

Percentage errors found: 57.05 
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Hcdule: 1 


Procedure No.: 76 



Number of nodes: 

15 

Number of arcs: 

19 

Number of paths: 

5 

Number cf sctrce stmts.: 

20 

Average errcr found: 

0.3893 

Percentage errors found: 

4 0.88 
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Module: 1 


Procedure Ho.: 77 



Hurler of nodes: 

17 

Hurler of arcs: 

20 

Nunler of paths: 

9 

Humber of source stmts.: 

10 

Average error found: 

0.2425 

Percentage errors found: 

50.93 


77 








Module 


1 


Procedure No.: 7S 



Number of nodes: 

25 

Number of arcs: 

31 

Number of paths: 

3 

Number cf sccrce stmts.: 

23 

Average error found: 

0.3628 

Eercentace errors found: 

33.13 
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Hcdule: 1 


Procedure No.: 81 



Number o f nodes: 8 

Number of arcs: 9 

Hunter of paths: 3 

Hunter cf scrrce stmts.: 7 

Average etrcr found: 0.1449 

Percentage errors found: 43.47 
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Module: 1 


Procedure No.: 86 



Number of ncces: 

18 

Number of arcs: 

23 

Number of paths: 

13 

Number cf source stmts.: 

22 

Average error found: 

0.3370 

Percentage errors found: 

32. 17 
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Module 


1 


Procedure No.: 87 



Number of nodes: 

21 


Number of arcs: 

22 


Number of paths: 

6 


Number cf source stmts.: 

25 


Average error found: 

0 . 

5029 

Percentace errors found: 

42 

.24 
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Module: 1 


Procedure No.: 91 



Number of nodes: 22 

Number of arcs: 30 

Number of patis: 9 

Number cf source stmts.: 14 

Average error found: 0.1438 

Percentage errors found: 21.57 
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Module: 1 


Procedure No.: 


92 



Number of rcces: 5 

Number of arcs: 6 

Number of paths; 3 

Number cf sctrce stmts.: 9 

Average errcr fcund: 0.1837 

Percentage exrcrs found: 42.86 
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Module 


1 


Procedure No 


93 



Hunter of nodes: 25 

Nualer of arcs: 34 

Number of paths: 12 

Number of source stmts.: 34 

Average error found: 0.3972 

Eercentace errors found: 24.53 
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Module 


1 


Procedure No. : 


95 



Nunher of ncces: 

18 

Nunher of arcs: 

27 

Nunher of paths: 

10 

Number of source stmts.: 

19 

Average error found: 

0. 1822 

Percentage errors found: 

20.14 
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Module: 2 


Procedure No.: 15 



Number of ncces: 

11 

Number of arcs: 

13 

Number of paths: 

5 

Number cf settee stmts.: 

12 

Average error found: 

0.0836 

Percentage errors found: 

35.5S 
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flcdule: 2 


Procedure No 


23 



Nuirber of ncces: 

10 


Nuiiber cf arcs: 

11 


Nunter of paths: 

3 


Number cf source stmts.: 

12 


Average errci found: 

0 . 

1592 

Percentage errors found: 

67 

.66 
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Module 


Procedure No.: 40 



Nunfer of nodes: 

22 

Nualer cx arcs: 

27 

Nuiler of jatfcs: 

5 

Nuttier of source stmts.: 

14 

Average error found: 

0-2018 

Percentage errors found: 

73.51 
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Hcd ule 


Procedure No 


4 1 



Nuaber of noces: 

10 


Nuaber cf arcs: 

14 


Nuaber of paths: 

12 


Nuaber cf sctrce stmts.: 

13 


Average errcx found: 

0 . 

1554 

Percentage encrs found: 

60 

.96 
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Module: 


Procedure No.: 46 



Number of patfcs: 36 

Number cf source stmts.: 34 

Average error found: 0.1657 

Percentage eircrs found: 24.86 
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Module: 


Procedure Mo.: 4 7 



Hunter of ncces: 

24 


Number of arcs: 

34 


Number of paths: 

36 


Number of sctxce stmts.: 

34 


Average error found: 

0 . 

1 163 

Percentage errors found: 

17 

.45 
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Module 


2 


Procedure Mo.: 48 



Hunter of ncces: 

16 


Number of arcs: 

21 


Hunter of paths: 

1 4 


Number of source stm.ts.: 

17 


Average error found: 

0 . 

1580 

Percentage errors found: 

47 

.40 


92 










Med ule: 


2 Procedure No.: 



Nunter of nodes: 18 

Nunter of arcs: 22 

Nuirter of jatbs: 10 

Nunter cf source stm.ts.: 34 

Average error found: 0,1885 

Percentage errors found: 28-28 


73 


93 


















Module: 


Procedure No.: 79 





Nuafcer of nodes: 

13 


Nuafcer cf arcs: 

14 


Nuafcer cf paths: 

5 


Nunfcer cf scnce stats.: 

11 


Average errci found: 

0 . 

1379 

Percentage errors found: 

63 

.94 


94 















acd ule: 


Procedure No. : 82 



Hunter cf nodes: 

23 


Hunter cf arcs: 

24 


Nuater of paths: 

2 


Hunter cf scarce stm.ts. : 

35 


Average errcr found: 

0 . 

1130 

Percentage errors found: 

16 

.47 


95 












Module: 


Procedure No.: 86 



Nuiiter of nodes: 

30 


Number of arcs: 

34 


Number of paths: 

6 


Number of sccrce stmts.: 

33 


Average error found: 

0 . 

2042 

Percentage errors found: 

31 

.56 


96 










Module: 


Procedure No.: 90 



1/1 


Number of nodes: 

13 


Number cf arcs: 

18 


Number of paths: 

8 


Number cf source stmts-: 

23 


Average error found: 

0 . 

0 95 8 

Eercentape errors found: 

21 

. 24 


97 















Module: 


Procedure No.: 99 


z 



Nunber of nodes: 

25 


Nuttier of arcs: 

30 


Nunier of paths: 

10 


Nuuber of source stm.ts.: 

23 


Average error found: 

0 . 

1513 

Percentage errors found: 

33 

.55 


98 














Module: 2 


Procedure No.: 122 



Nuafcer c£ nodes: 

18 


NuiBler cf arcs: 

21 


Nuater of paths: 

6 


Nuater cf sctrce stats.: 

20 


Average error found: 

0 . 

1686 

Percentage errors found: 

42 

.99 


99 







Module 


Procedure No.: 137 



Nuater of ncces: 

13 


Number of arcs: 

17 


Nuirher of paths: 

1 1 


Number of source stmts.: 

23 


Average error found: 

0 . 

1178 

Percentage errors found: 

26 

. 12 


100 












Med ule: 


Procedure No.: 149 



Nuaher of 

nodes: 

18 

Number cf 

arcs: 

25 

Hunter of 

paths: 

9 

Number cf 

source stmts.: 

35 

Average errer found: 

0.2357 

Eercenta ce 

errors found: 

34.34 


101 
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